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Abstract 


Let  y  =  o  +  x’3  +  e,  i  -  1 . n,  ...  be  a  linear  regression 

i  1  1  -  ' 


fnodel. 


where  (x()  is  a  sequence  of  experimental  points,  i.  e.,  known  p-vectors.  (e)  is  a 
sequence  of  independent  random  errors,  with  medley  =  0,  i  =  1.  2 . Define  the 

|  A  A 

minimum  L  ^  -norm  estimate  of  ^dt,  BT.  by  (pn,  Bn)’,  to  be  chosen  such  that 

^  \\ 


J.  K  -  «n  •  Xi?nl 


■  min  l  |y.  -  a  -  x|0| 

a.B  ‘*i 


Under  quite  general  conditions  on  tx,}  and  (ej),  the  strong  consistency  of  the 

minimum  L  -norm  estimate  is  established.  Further,  under  an  additional  condition  on 

t  *  •- 

(Xj),  it  is  also  proved  that  for  any  given  $  >  0,  there  exist  constant  C  >  0  not 
depending  on  n.  such  that 


P(||a  -  o||2  ♦  ||Bn  -  B||2  >  e2) 


<  exp{-Cn},  for  large  n. 


AMS  1980  subject  classifications:  Primary  62J05;  Secondary  62F12. 

Key  words  and  phrases:  L'j-norm  estimation,  consistency,  exponential  rate, 

linear  model,  median  regression. 


1.  Introduction. 


Consider  the  linear  regression  model 

y.  -  a  +  x!p  +  e. .  i  -  1,  2,  .  .  . 

(1.1) 

where  {  x.  =  (x.^  .  .  .  ,  x,J  },  i  =  1,  2 . is  a  sequence  of  experiment  points, 

i.  e.  known  p-vectors,  $  =  (0  is  the  regression-coefficient  vector,  and 

_  i  P 

e  ,  e  ,  .  .  .  are  random  errors.  To  estimate  the  unknown  vector  B  based  on  the 
i  2  Z 

observations  y  ,  .  .  .  ,  y  ,  a  popular  method  is  the  Least  Squares  (LS)  method,  which 

1  n 

-  n 

2 

takes  B  minimizing  the  sum  of  squares  £  (y.  -  x'B)  as  the  estimate  of  B 

i  \ 
i*  1 

The  merit  of  LS  estimate  is  that  it  is  easy  to  compute  (being  a  linear 
combination  of  y . y  ),  and  that,  in  case  where  the  random  errors  are 

1  n 

independently  and  approximately  normally  distributed,  the  LS  estimate  possesses 
many  desirable  properties.  But  these  happy  conditions  cannot  be  taken  for  granted  in 
many  practical  applications.  For  example,  in  econometrics,  there  now  exists  a 
considerable  body  of  evidence  that  attests  to  distributions  with  infinite  variance 
being  a  reality  (distribution  of  income,  behavior  of  speculative  prices,  distribution  of 
firms  by  size  etc.).  Even  in  cases  where  the  error  variance  can  reasonably  be 
assumed  to  be  finite,  the  error  distributions  may  have  heavy  tails,  deteriorating  the 
performance  of  the  LS  estimate. 

So  it  is  under  this  background  there  has  now  been  much  interest  in  using 
more  robust  methods,  among  which  the  L  i  -norm  method  is  a  forerunner.  This 

n  n 

2 

method  seeks  to  minimize  T  |y  -  x’B  I  instead  of  V  (y  -  x'B)  ,  and  the 

u  1  i  i  i 


2 


minimizer  g  is  taken  as  the  estimate  of  g.  True,  the  calculation  of  £  is 

considerable  more  complicated  as  compared  with  the  case  of  LS  estimate  g  ,  but. 

since  the  successful  establishment  of  the  link  between  the  L  i  -norm  method  and  the 
linear  programming,  the  computing  problem  no  longer  presents  a  barrior  in  the  face 
of  modern  computing  facilities. 


Another  problem  is  the  sampling  theory  of  the  estimate.  Since  it  is  not  likely 
that  a  workable  small-sample  theory  can  be  established,  the  asympototic  theory  is 
of  great  importance.  The  first  and  foremost  problem  in  the  asympototic  theory  is  to 
establish  the  consistency  of  this  estimate  under  weak  conditions. 

The  asymptotic  theory  of  the  l^-norm  estimate  is  much  more  difficult  as 
compared  with  the  Least  Squares  theory,  owing  to  the  mathematical  difficulty  of 
working  with  the  absolute  value  function.  Huber  (1981)  proved  a  general  theorem 
concerning  the  consistency  and  asymptotic  normality  of  a  class  of  robust  estimates 
of  linear  regression  coefficients,  but  the  result  does  not  apply  to  the  L  i  -norm 
estimate,  since  the  absolute  value  function  is  not  differentiable  at  zero.  In  recent 
years  some  authors,  for  example  Oberhofer  (1982),  proved  the  weak  consistency  of' 
the  1 1 -norm  estimate  under  rather  strong  conditions. 

In  this  paper,  we  develop  a  method  in  dealing  with  the  consistency  problem 
of  the  L  i  -norm  estimate  which  enables  us  to  obtain  general  results  concerning  the 
strong  consistency  and  exponential  rate  of  the  L  i  -norm  estimate  under  very  mild 
conditions. 


2.  Formulation  of  the  Theorems. 

In  some  applications  it  may  be  known  in  advance  that  the  regression  plane 
passes  through  the  origin,  i.  e.  a  =  0  in  model  (1.1).  In  this  case,  according  to  the 

a 

L  -norm  criterion,  the  solution  g  of  the  minimization  problem 

1  _  n 


rewire 


gives  an  estimation  of  B  Define 


n 

S  -  l  x.x! , 

n  i-i  '  ' 

p  -  the  smallest  eigenvalue  of  S  , 
n  n 

d„  -  max  {  1,  HxJI.  ...  ,||xnl|  }, 

where  1 1  x  1 1  denotes  the  Euclidean  norm  of  vector  x. 

Theorem  t.  Suppose  the  the  following  conditions  are  satisfied: 
2 

1)  p  /(d  logn)  oo, 
n  n 

2)  There  exists  constant  k  >  1  such  that 

.  ,  k-1 
d  /n  -►  0, 
n 

3)  e^  e  ,  .  .  .  are  independent  random  variables,  and 
med  (e  )  -  0,  i  ■  1,  2,  .  .  . 

4)  There  exist  constants  C  >  0,  C  >0  such  that 

\  2 


a.  s. 


lim  Bn  -  B, 

n-v<=  ~ 

Further,  under  the  additional  condition  that  for  some  constant  M  >  0 

2 

p  /d  >  Mn,  for  large  n 
n  n  — 

(2.5) 


B  converges  to  B  at  an  exponential  rate  in  the  following  sense:  For  arbitrary  given 
c  >  0  there  exists  constant  C  >  0  independent  of  n  such  that 

'  B||  >  e) 


_ .  -Cn. 

<  O  (e  )  . 

Theoretically  speaking,  the  nonhomogeneous  model  (1.1)  is  merely  a  special 

case  of  the  homogeneous  model  y  =  x'  B  +  e ,  i  =  1 ,  2,  ,  in  which  first 

i  i_  i 

element  of  each  x  is  1.  Therefore,  as  a  corollary,  from  Theorem  1  we  can  obtain 

A  A 

an  analogous  result  concerning  the  estimates  a,  B  of  a,  B:  We  need  only  to 

n  ~p 
n 

replace  the  matrix  =  £  x  x'  by 


and  define  p  as  the  smallest  eigenvalue  of  S  .  But,  since  S  is  a  matrix  of  higher 

n  n  n 

order  as  compared  with  S  ,  it  may  be  of  some  advantage  to  give  the  following 

n 

result 


Theorem  2.  Suppose  that  the  we  have  model  (1.1),  and  the  conditions  of 

n 

Theorem  1  are  satisfied,  except  that  here  we  define  S  as  7  (x  -  x  )  (x  -  x  )’ 

n  “  i  n  i  n 

i=1 

n 

where  x  =  (1/n)  7  x,  then 

n  “  i 

i*  1 


1  i m  a  *  a»  a.  s . 
n 

n-*.co 


1 im  g  ■  3 ,  a.  s . 
_n 

n-*.oo 


Also,  under  the  additional  assumption  (2.5)  for  arbitrarily  given  e  >  0  we  can  find 
constant  C  >  0  independent  of  n  such  that 

-  a||2  +  ||^  -  B||2  >  c2) 


_  -  -Cn. 
O  (e.  ) 


In  this  formulation,  Theorem  2  is  not  a  trival  consequence  of  Theorem  1,  and 


,  i  ♦. »  *_»»,*  >.i  it>  i.ft.i  i.i  i  i 


' . i> i  i-i  «*•  i  i 


/I  4k*  «»*<** 


.  •-» 


through  the  basic  idea  of  proof  are  the  same  for  the  two  theorems,  some 
important  differences  in  detail  emerge.  Therefore  we  shall  give  the  detailed  proof  of 


3.  Comments  on  Conditions. 

Before  entering  the  details  of  proofs,  we  make  some  remarks  concerning  the 
conditions  of  the  theorems. 

1.  The  first  two  conditions  involve  only  the  sequence  of  experiment  points 
M.  while  the  latter  ones  involve  only  the  error  sequence  {e}.  In  any  theoretical 
problem  concerning  the  linear  model  (1.1),  it  is  always  desirable  not  to  introduce 
assumptions  involving  both  simultaneously. 

2.  Conditions  3)  and  4),  taken  together,  guarantee  the  uniqueness  of  the 

median  of  e.,  i  =  1,  2 . Condition  4)  stipulates  that  the  random  errors  (es) 

should  not  be  "too  lightly”  distributed  around  the  median  zero.  The  requirement  on 
the  uniqueness  of  median  is  reasonable  in  view  of  the  "median-regression"  character 
of  the  model.  As  for  the  conditions  (2.3)  and  (2.4),  it  is  likely  that  they  are  not 
necessary  and  further  improvements  are  conceivable,  yet  it  is  easily  seen  that  they 
cannot  be  totally  dispensed  with. 


Example  1.  Take  the  simplest  case  in  which  we  know  in  advance  that  g  =  0. 
In  this  case  the  Minimum  l  -norm  principle  gives 


a  ■  med  (y  ,  .  .  .  ,y  ) 
n  i  n 


as  the  estimate  of  a  Suppose  that  e^,  .  .  .  are  mutually  independent,  e  has  the 


0, 


otherwi se. 


Then 


P{e.  >  0  -  1/2  -  l/(2i2)  ,  i  -  1,  2,  .  .  . 

Denote  by  £  the  number  of  those  e's  for  which  /n  <  i  <  n  and  e  >  1.  An 

n  i  —  ~  i  — 

application  of  the  Central  Limit  Theorem  convinces  us  that  for  given  6  e  (0,  1/2) 
we  have 

PH  >  n/2}  >  6 
n 

for  n  sufficiently  large.  This  implies  that 

P(a  >  1}  >  6 
n 

for  n  sufficiently  large,  hence  a  is  not  a  consistent  estimate  of  a. 

n 

3.  Conditions  1)  and  2)  regulate  the  behavior  of  the  sequence  of  experiment 
points  {x  }.  2)  stipulates  simply  that  x  should  not  go  to  infinity  "too  fast”  as  n 

n  n 

A  close  inspection  of  condition  1)  convinces  us  that  this  is  also  implied  by  1). 
Condition  1)  requires  that  p  should  tend  to  infinity  with  some  rate.  In  the 

n 

case  where  {x}  is  bounded,  p  should  tend  to  infinity  at  a  rate  faster  than  logn.  In 
the  LS  theory,  under  some  general  conditions  (see  Li(1 984))  on  the  error  sequence 
{e},  the  strong  consistency  of  LS  estimates  is  guaranteed  by  requireing  only  p  -► 

*  n 

w.  This  gives  one  the  hope  that  condition  1)  can  be  weakened  to  p  -►  oo.  Whether 

n 

or  not  this  is  true  remains  an  open  question. 

The  point  that  "x  should  not  go  to  infinity  too  fast"  is  of  interest  as  it 


reveals  a  difference  between  the  L  i  -norm  and  LS  criteria  For  LS  estimate,  in 
general  the  faster  x  goes  to  infinity  as  n  +  ®,  the  more  likely  is  that  it  becomes 

n 

consistent  The  following  example  shows  that  in  the  l^-norm  case,  going  too  fast 
(of  x  )  to  infinity  may  indeed  render  the  estimate  inconsistent 

n 

Example  2.  Suppose  that  in  model  (1.1)  p  =  1,  the  random  errors  e^, 

e2<  .  .  .  independent,  P{e  =  1  o'}  =  P{e.  =  —  1 01}  =  1/6,  and  e  is  uniformly 
distributed  over  the  interval  (-1/3,  1/3)  with  density  one.  For  convenience  assume 
that  the  true  parameter  values  are  a  =  0,  3  =  0-  Let  x.  =  10  ,  i  =  1,  2 . 


Define  the  event  E  =  {e  =  10  }.  Then  P{E  }  =  1/6.  It  is  readily  seen  that 

n  n  n 

when  E  occurs  we  have 

n 


£  |e.  -  a  -  bx. |  >  4x10°  ,  when  |a|  <  1/10  and  |b|  <  1/10, 


7  |e.  -  a  -  bx.  I  <  3x10 

1  i  i 'a-O.b-l  “ 


These  two  facts,  taken  together,  give 


P{  max  ( | ot  I,  |3  |)  >  1/10}  >  P{e  }  -  1/6. 
1  n 1  'n 1  n 


So  even  weak  consistency  does  not  hold. 


Example  3.  This  example  shows  that,  even  in  the  case  that  e^,  .  .  .  are 
independent  and  identically  distributed,  consistency  may  not  hold  in  case  p  tends  to 

n 

infinity  too  fast 


Suppose  that  in  model  (1.1)  p=1,  the  true  parameters  a  =  0,  3  =  0,  the 

k  k 

random  errors  are  iid.  with  a  common  distribution  P{e  =10}=P{e  =—10}  = 


1/[k(k+1)],  k  =  6,  7 .  and  e  is  uniformly  distributed  over  (-1/3,  1/3)  with 

density  1.  Let  x  =  1 0*.  i  =  1.  2,  .  .  . 

Define  the  event  E^  =  {|e.|  <  10n,  n/2  <  i  <  n}.  Then  it  is  readily  seen  that 
-2 

P(E  )  -*.  e  as  n  +  ®.  Suppose  that  E  does  not  occur  and  denote  by  i  the  first  j 

n  n  n 

such  that  j  >  n/2  and  |e  |  >  10n,  then  we  have  i  <  n.  An  argument  similar  to  that 

A  A  A 

employed  in  Example  2  gives  us  that  max  ( |  a  | .  |  p  | )  >  1/10.  Therefore  P  { |  a  |  > 

n  n 

1/10  or  Ip  I  >  1/10,  for  some  i,  n/2  <  i  <  n}  >  P{EC}  -►  1  -  e  2  >  0  and  (a,  P) 

1  i 1  —  —  —  n  n  n 

is  not  strongly  consistent 

From  a  practical  point  of  view  it  can  be  said  that  the  conditions  1)  and  2) 
are  reasonable  and  would  be  satisfied  in  most  applications.  An  important  case  is  that 
(x  }  is  bounded,  or  more  generally,  J  |  x.  1 1  -  o(i/logi),  and 

lim  S  /n  ■  A  exists  and  positive  definite, 
n 

n+co 

(3.1) 


These  conditions  are  satisfied  when  x  ,  x  ,  .  .  .  are  iid.  samples  of  a  random 

1  2 

vector  x  for  which  E(xx')  exists  and  is  positive  definite. 


4.  Proof  of  Theorem  1. 

The  following  lemmas  will  be  needed  in  the  proof. 

Lemma  1  (Bennett.)  Suppose  that  £,...,£  are  independent  random 

1  n 

n  n 

variables  with  |$.|  <  b  <  ®.  Let  vi  =  (1/n)  £  E£  and  o2  =  (1/n)  £  Var£.  Then 


10 


I 


p{|  (1/n)  l  5  -  v\  >  e) 

i*  1 


<  2exp{-ne2/[2  (o2  +  be)]}. 


The  proof  of  this  lemma  can  be  found  in  Hoeffiding  (1963). 

Lemma  2.  Suppose  that  {e.},  i  =  1.  2 . n  are  random  variables 

satisfying  the  conditions  3)  and  4)  of  Theorem  1.  Also,  let  {a.},  i  =  1 . n  be  a 

sequence  of  constants  and  a*  =  max  j  a  | .  Then  there  exist  constants  C  >  0,  and 

i<i<r 

eQ  >  0,  depending  only  on  a,  and  C^,  such  that 


HE  (|.,|  -  le.  -».|>  >-e„Bn> 

I*  1 


<  2exp{-CB  } , 
“  n 


where  B  =  7  a  . 

n  j 

i*  t 


(4.1) 


Proof.  Without  loss  of  generality,  we  can  assume  that  a.  >  0,  for  each  i  =  1, 

.  .  .  ,  n.  Note  that  (4.1)  is  automatically  true  if  a*  =  0.  Thus  we  can  assume  that  a  > 
0,  which  implies  that  B  >0.  In  case  of  a  >  C  ,  we  can  suitably  adjust  C  such 

n  1  2 

that  (2.3)  and  (2.4)  are  true  for  each  i  and  h  e  (0,  a*). 


Define  £  =  |e  |  -  |e  -  a. |.  It  is  easy  to  see  that 


-a. , 

i 

if  e.  <  0 

• 

a 

CM 

if  0  <  e.  <  a. . 

t  i 

U;  I  <  <  a  . 


EC-  <  -a.p{e.  <  0}  +  a.P{e.  >  a./2}, 

i  i  i  i  i  i 


<  -a.P{0  <  e.  <  a./2}  <  -C,af/2, 

i  i  i  2  i 


Var C.  <  EC,  <  a2 


Take  eq  =  C2/4  and  C  =  e*  /[2(1  +  a*eo0  Using  Lemma  1,  we  have 


p{ 


P{ (1/n)  l  (C.  -  E?  )  >  -(1/n)  (  l  EC.  +  e.B2)} 

4  I  ■  .  I  V  n 


i  P{  O/n)  l  (C.  -  EC,)  >  EQB7nJ 


<  2exp{-n(enB2/n)2/[2(B2/n  +  a*e„B2/n)]} 
on  n  On 


•  2exp{-CB  }, 
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which  comletes  the  proof  of  Lemma  2. 

Lemma  3.  Maintaining  the  assumptions  and  notations  given  in  Lemma  2.  and 
defining 

r  1.  if  e.  >  a./2 

'  i  i 

*1  * 

-1 ,  if  e.  <  a./2, 

i  —  i 

we  have 

n  0  0 
p{  l  a.Ti.  >  -e0Bn}  <  2exp{-CB^} . 

i»  1 

The  proof  runs  largely  along  the  same  line  as  in  Lemma  2.  So  the  details  are 
omitted 

Now  turn  to  the  proof  of  die  theorem  Without  loss  of  generality  we  assume 
that  the  true  parameters  0  =  0. 

mm 

Fix  e,  0  <  e  <  1.  and  define  A  =  {0:  ||0||  -  e}-  Split  A  into  m  =  m 

mm  O 

parts  A, . A  such  that  the  diameter  of  each  part  does  not  exceed  e/nk.  It  is 

1  m 

easy  to  see  that  this  can  be  done  with 


Pk 


m  •  m  <  (pn)  . 
n  * 


Choose  arbitrarily  a  point  0t  from  each  A{,  t  =  1 . m  Define  b{  =  x’0  . 


b  =  max  {  b  ,  1  }  and  a  =  b  /b .  Then  we  have  a  <  1  for  each  t  and  i. 

t  1  ti 1  ti  ti  t  1  ti  1  - 

i<j<p 

Also,  we  have 


Btn  ■  r  *,?  ■ 

IB  1  “  “ 


a 


5 


i 


a 
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>  p  /d  +  co. 
-  Kn  n 


Hence,  by  Lemma  2,  we  have,  for  targe  n. 


l  d«,l  -  I*,  -  \,l>  >  -«> 

i«  1 


<  Pi  I  (|«,l  -  I*,  -  «ti|)  >  -«Btn> 


<  2exp{-CB  <  2expi-Ce*p  /d*}. 

—  tn  •  n  n 


If  B  e  A  and  Y  <  I e  —  I e  —  a  I >  <  ~e,  then 

t  **  1  i  1  t  ti  ■  — 


I  <IM  ‘  I*.  -  xl?/btl> 


<  I  (|.  |  -  |«  -  .  |>  ♦  l  |xMB  -  »  )|/b 

i“  1  i«1 


<  -e  +  nd  e/n  <  0,  for  all  large  n. 
""  n 


Here  the  last  step  follows  from  condition  (2.2).  Therefore,  for  all  large  n  we  have 


P{  l  |e  |  -  min  £  |e  -  x!B|  >  0} 
i-i  BcA^ 


<  ?yi  (i«,i  - 1*,  -  »tiP  > 


w 
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<  2exp{-Ce2p  /d2} 
“  n  n 


where  A  =  {b  0:  $  e  A } 

t  t  _  _  t 


Now  let  A  >  1.  Define 


if  e.  >  a  ./2 

i  1 1 


1,  if  e.  <  a  . /2 

i  1 1 


i  -  1.  .  .  .  ,n 


A.  «  I e . |  -  |e.  -  £a.  | , 

i  1  i  1  1  i  i  1 

where  a  =  x'0,  0  e  A'  .  Let  us  consider  the  following  cases.  In  each  case,  we 
•  _  t 

shall  use  the  fact  that  la  -  a  I  *  I  x’  10  -  0  >  |  /b  <  d  c/nk  =  o(1/n).  Here  and  in 

1  ti  i 1  1  i  „  _t  t  n 

the  sequel,  o<1/n)  is  uniform  in  t  and  i. 


Case  a.  a  >  a  /2  >  0  and  e  <  a  /2. 

i  -  ti  i  ti 


We  have 


A.  -  |e. |  -  |e.  -  a, |  -  (A 

i  1  i 1  1  i  i  1 


-  1)  a. 


<  l«;l  -  I  * :  -  a.  |  ~  (A  '  I)  (atl  -  o(1/n)) 


|e.  |  -  |e.  -  a. |  +  (A  -  1)  (a^.T^.  +  oO/n)) 


Case  A.  a  >  a  /2  >  0,  and  e  >  a  12. 

i  -  ti  i  -  ti 


! 


5 

»• 

•« 

s 


r 

S1 


l 

\\ 

►I? 


§ 


5 


We  have 

A.  <  |e. |  -  |e.  -  a. I  +  (ft  -  1) a. 

,  -  i  i i  i  |  i 

<  |e.  |  -  |e.  -  a.  I  +  (ft  -  I)  (a  +  o(l/n)) 

-  i  ,  i  i  i  ti 

-  | e .  |  -  |e .  -  a.  |  +  (ft  -  1)  (a t } n t .  +  o(  1/n))  , 

which  shows  that  (4.3)  is  still  true. 

Case  c.  0  >  a  /2  >  a  and  e  >  a  /2. 

ti  —  i  i  ti 

We  have 

A.  ■  |e. I  -  |e.  -  a. I  +  (ft  -  1) a. 

i  1  i  1  1  i  i  1  i 

<  |e.|  -  |e.  -  a.  |  +  (ft  -  1)  (at.  +o(1/n)) 

-  I«jl  "  l«;  ~  «j|  +  (A  '  0  («tiTlt.  +  o(l/n) )  . 

Hence  (4.3)  is  still  true  for  this  case. 

Case  d  0  >  a  12  >  a  and  e  <  a  /2. 

t»  —  i  i  ~  t» 

We  have 

A;  <  |*;|  -  |«;  -  »j|  +  (ft  -  1) I»j| 

5  l«;l  -  l*f  *  a{l  +  (A  -  1)  (|»t . I  +  °(l/n) ) 

"  lei  I  -  lei  -  «.  |  +  (A  -  1)  (atihtj  +  o(  1/n)) , 

which  also  shows  that  (4.3)  is  true. 


ft,  £*_*» 


**■**■»« 


,1.  ,l 


Case  e.  ja.|  >  |  |  /2. 


In  this  case,  we  have  that  la  I  <  2 1  a  -  a  I  =  oO/n)  hence  a  =  o(  1/n). 

1  ti  1  -  1  •  ti  i  1  i 


Therefore 


A.  -  |e. |  -  |e.  -  Aa. | 

i  1  i  1  1  »  i  1 


<  |e.  |  -  |e.  -  a.  |  +  (A  -  l)o(l/n) 


-  |e.  |  -  |e.  -  a.  |  +  (A  -  1)  (a  n  .  +  o(l/n))  , 

'  i 1  1  i  i 1  ti  ti 


and  (4.3)  is  still  true. 


Summing  up  the  five  cases,  we  see  that 


A,  <  |e.  |  -  |e.  -  a.  |  +  (A  -  1)  (at.Tit.  +  o(l/n)) 

Let  11'  =  (A&:  0  e  A’  and  A  >  1}  .  By  (4.2),  (4.3)  and  Lemma  3,  we  have 

^  am  m»  ^ 


P{  l  |e  |  -  min  J  |e  -  x|0|  >  0} 
*-»  BeJIJ  i-i 


<  p{  l  l«j I  -  min  I  I®,  -  *;&l  >  o> 

..i  0eA^  i-i 


* r{  J.  *ti\i  i  •*> 


<  4dxp{-Ct*p  /d*} . 

n  n 


Denote  II  =  {AB  0  c  A  and  A  >  1}.  Note  that  b  >  1.  we  have  JIC  IT 

t  «.  _  t  -  t  -  t  t 

m  m 

Also.  { 1 1 B 1 1  >  e)  *  U  n  C  U  II  Hence  we  obtain 
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.*  't 


P{  l  |e  |  -  min  £  |e.  -  x|B|  >  0} 

-i  l|B||>e  «-i 


n  n 


5  I  P*  I  le: I  l  le:  *  *|B|  >  0} 

t-1  i-1  RelT*  i-1 


Ben; 


<  4mexp{-Ce2p  /d2} 
n  n 


<  A  (pn)  pkexp{-Ce2p  /d2} 

n  n 


which,  together  with  (2.1)  and  Borel-Cantelli  Lemma,  implies  that 


P{|  |Bj  |  >  e.  i  •  o.}  -  0 


This  proves  the  strong  consistency  of  B  -  Finally,  the  last  assertion  of  the  theorem 
follows  directly  from  (2.5)  and  (4.4).  The  proof  is  concluded. 


Remark.  If  (3.1)  holds,  then  condition  (2.2)  for  k  =  3/2  is  fulfilled.  If  we 
further  assume  that  d^  =  o(/n/logn),  then  (2.1)  is  satisfied.  Also,  it  is  easy  to  see 
that  (3.1)  and  the  boundedness  of  d  imply  (2.2)  and  (2.5)  which  guarantee  the 

n 

exponential  rate  of  the  convergence  of  the  L  i  -norm  estimate.  It  may  be  noticed  that 

2  _cn 

since  in  general  we  have  p  <  nd  ,  we  cannot  get  a  rate  faster  than  CXe  )  from 

n  —  n 

the  proof  of  Theorem  1.  In  fact,  under  the  assumption  that  e^  .  .  .  are 

A 

independent  and  identically  distributed,  it  can  easily  be  shown  that  P  { |  ( B  ~  B 1 1  > 
e)  cannot  tend  to  zero  at  a  rate  faster  than  0(e  )  (i.  e.  P{||^  —  @||  >  e}  = 

0(g(n))  and  eCng(n)  0  for  any  C  >  0  as  n  ®) 


>  .V .  'J'J: 


5.  Proof  of  Theorem  2. 


Without  loss  of  generality,  we  can  assume  that  a  =  0,  &  =  0. 
Given  e,  0  <  e  <  1,  and  define 

A  -  {(a,  :  a2  +  g'g  -  e2) . 

Split  A  into  m  =  m  parts  A  ,  A  such  that  the  diameter  of  each 

n  1  m 

not  exceed  e/nk  and  that  m  <  (to  +  1)n)<P+  1>k  =  C  ntp+1>k  where  C  is 

oo 

constant  Choose  arbitrarily  a  point  (at>  fT  )'  from  A{  and  define 


b 


ti 


Consider  the  following  two  cases. 

Case  1.  b  <  e/4. 

t  - 

Define  a  =  a  +  b  .  Then  we  have 

ti  t  ti 

Iatil  i  t  +  e/l*  <2,  i  ■  1 ,  ...  ,n. 
Arguing  as  in  the  proof  of  Theorem  1,  we  get 


part  does 
a  positive 
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<  2exp{-C  l  a  }  , 
i«i  11 


and  consequently 


fiE  Kl  - 


min  I  |e.~  A (a+x! g)  I  >0} 
(a.g')'eA  i-i  ' 


<  2exp{-C  l  a  } . 
i*  1 


Noticing  that  <  e/4,  we  have 


j,  *!i-  n(at  +  w2  + 


>  n (e/2  -  e/4)2  -  ne2/l6. 


Hence,  by  (5.1)  and  (5.2)  we  have 


»■  n 

PC  l  |e.  |  -  min  J  |e.-  £(a+x!g)|  >  0} 
(a.B')'eA  i-i  1 


<  2exp{-Cn} 


Case  2.  b  >  e/4. 


Define  a  =  e(a  +  b  )/(4b).  We  have 

ti  t  ti  t 


>: 


lati  I  1  e  +  eA  <  5A< 


As  before,  using  Lemma  2,  we  can  show  that 


l  (|..|  -  I',  -  .ti|>  >  -c) 


<  2exp{-C  \  a  > . 

i*  1 


For  (a,  B')'  e  A(,  define  a.  =  e(a  +  x‘.B)/(4b{).  As  in  the  proof  of  Theorem  1 


we  have 


P{  l  | e  |  -  min  I  |«:  " 


(o.B1)  '  eA 
“£>1 


Aa. I  >  0} 

i  1  - 


<  Lexp{-C  l  a  } . 

i*  1  11 


Noticing  that  c/(4b^)  <  1,  we  have 


(UtAbt)  (a,  B ')  1 :  (a.  B ' )  '  e  At>  i  > 


^  tt(a,  B1)  (a,  B')  '  e  A  .  i  >  1}. 


Hence,  by  (5.4)  and  (5.5)  we  obtain 


PC  I  I*.  |  "  min  l  |e.  -  &(ot+x!B)  |  >  0} 
(o.B')'eA  ‘-1 


Vn 


v>> 


(5.6) 


n  2 

<  4exp{-C  l  a  } . 

i=  1  11 


Since 


l  a  -  (e/4b  )2  l  («  +  x|B.)2 

i*  1  i=1  1  ~ 


>  e2/  06b2)  p 'T  B„  >  ep  /(l6d2), 
t  _  t  n_t  n  n 


we  have,  by  (5.6), 


n  n 

p{£  I  e .  |  -  min  £  |e.-  A (a+xl B) |  >  0} 

'  (a.B')'eA  *-1  ' 

~A>1 


<  4exp{-Cp  /d2} , 
n  n 

(5.7) 


where  C  >  0  is  a  constant  independent  of  n,  but  may  be  taken  as  different  value  at 
each  appearance. 


From  (5.3)  and  (5.7),  we  finally  obtain  that 


P{  I  |e.  |  -  min  I  |e  -  a  -  x!B|  >  0} 
a  +B1  B>e  im1 


<  M(p  +  l)n)  kexp{-Cp  /d2} . 

n  n 


In  view  of  condition  (2.1),  applying  Borel-Cantelli  Lemma  we  get 


a  -*•  0,  a.  s. 
n 


8  -►  0,  a.  s. 
_n 


These  prove  the  strong  consistency  of  a  and  £ .  The  last  assertion  of  the  theorem 
follows  from  (2.5)  and  (5.8).  The  proof  is  complete. 
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